Recursion and Probability

We discuss recent work on the algorithmic analysis of systems involving recursion and probability. Recursive Markov chains extend ordinary finite state Markov chains with the ability to invoke other Markov chains in a potentially recursive manner. They offer a natural abstract model of probabilistic programs with procedures, and generalize other classical well-studied stochastic models, eg. Multi-type Branching Processes and Stochastic Context-free Grammars. Recursive Markov Decision Processes and Recursive Stochastic Games similarly extend ordinary finite Markov decision processes and stochastic games, and they are natural models for recursive systems involving both probabilistic and nonprobabilistic actions. In a series of recent papers with Kousha Etessami (U. of Edinburgh), we have introduced these models and studied central algorithmic problems regarding questions of termination, reachability, and analysis of the properties of their executions. In this talk we will present some of the basic theory and algorithms. ⋆ Research partially supported by NSF Grant CCF-4-30946.